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ABSTRACT 


THE  RISK  OF  CAROTID  ENDARTERECTOMY  IN  THE  ELDERLY 
AN  APPLICATION  OF  CLAIMS  BASED  RESEARCH 


Neil  Andrew  Solomon 
1990 


Although  carotid  endarterectomy  is  commonly  performed,  controversy 
exists  over  its  risk  relative  to  its  benefit.  To  elucidate  risk  factors  for 
surgical  complications,  I  reviewed  Medicare  claims  files  for  2089  New 
England  residents  over  the  age  of  65  who  underwent  carotid  endarterectomy 
in  1984  and  1985.  Patients  over  the  age  of  70,  who  comprised  two  thirds  of 
the  sample,  faced  an  average  three-fold  increased  risk  of  operative  death 
compared  to  younger  Medicare  patients.  Also,  a  patient  was  almost  three 
times  more  likely  to  die  after  undergoing  carotid  endarterectomy  at  a 
hospital  performing  few  of  these  operations  relative  to  high  surgical  volume 
facilities.  Similarly,  the  risk  of  post-operative  stroke  was  directly  related  to 
age  and  inversely  related  to  surgical  volume.  The  identification  of  these  risk 
factors  helps  to  clarify  the  settings  under  which  carotid  endarterectomy  may 
offer  more  threat  than  benefit.  Further  research  is  needed  to  clarify  the 
causes  of  the  identified  correlations.  This  study  also  evaluates  the  ability  of 
the  claims  data  to  review  outcomes  of  the  operation. 
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Introduction 


Carotid  endarterectomy  surgery  offers  one  treatment  strategy  to  reduce 
the  likelihood  of  stroke  in  patients  with  atherosclerotic  disease.  The  surgery, 
however,  has  never  been  evaluated  in  a  satisfactory  way  to  clarify  who,  if 
anyone,  stands  to  benefit  from  this  approach  compared  to  medical 
management.  Factors  that  must  be  considered  in  balancing  the  risks  and 
benefits  of  the  operation  include  its  efficacy  and  its  complication  rate.  In 
this  study,  risk  factors  for  surgical  complications  are  examined  and 
subsequently  considered  in  the  medicine-versus-surgery  controversy. 

Stroke:  the  scope  of  the  problem 

While  the  incidence  of  stroke  has  declined  in  the  United  States  in  recent 
decades,’  it  remains  a  widespread,  debilitating  and  expensive  medical 
problem.  Today,  stroke  is  the  third  leading  cause  of  death  in  the  United 
States,^  behind  only  cardiovascular  disease  and  cancer.  In  1980,  more  than 
150,000  Americans  over  the  age  of  65  died  of  stroke.^  The  disability 
associated  with  the  1.9  million  non-fatal  events  annually,'*  including  paralysis, 
sensory  deficits,  and  aphasias,  adds  substantially  to  the  collective  emotional 
and  financial  burden.  The  greater  than  $5  billion  cost  of  stroke  care  in  the 
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US  each  year®  reflects  the  enormity  of  the  problem.  Because  there  exists  no 
proven  therapy  for  strokes  once  they  have  occurred,  the  major  thrust  of 
treatment  consists  of  prevention. 

Atherosclerosis  in  the  extracranial  arteries  causes  over  half  of  all 
strokes,^  and  approximately  75%  of  these  cerebral  infarctions  occur  in  the 
carotid  distribution.®  Other  causes  of  stroke,  including  cardiac  embolus, 
subarachnoid  hemorrhage,  intracranial  hematoma,  infection,  arteritis,  and  all 
events  in  the  vertebral-basilar  system,  are  not  potentially  correctable  by 
endarterectomy  and  therefore  fall  outside  the  scope  of  this  study. 

Risk  factors  for  atherosclerotic  stroke 

Several  risk  factors  for  atherosclerotic  strokes  including  hypertension,^ 
diabetes,®  and  hyperlipidemias^  have  been  identified.  Lifestyle’”"'^^  and  diet’® 
changes  in  the  population  appear  to  have  contributed  to  the  declining  stroke 
rate.  Even  with  improved  control  of  these  chronic  risk  factors,  a  large 
number  of  individuals  remain  at  elevated  risk. 

Two  large  epidemiologic  studies  associated  asymptomatic  cervical  arterial 
bruit  with  increased  likelihood  of  subsequent  stroke,’^’’®  indicating  a  risk  of 
2  to  4  times  over  that  of  persons  without  audible  bruit.  Both  research  teams 
concluded,  however,  that  the  bruit  is  a  non-focal  sign  that  does  not  correlate 
well  with  the  laterality  of  subsequent  cerebral  events.  Moreover,  death  in 
both  cohorts  was  most  likely  due  to  heart  disease,  not  stroke.  Similarly,  a 
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third  study  demonstrated  a  4%  annual  incidence  of  neurologic  symptoms  in 
patients  with  a  mid-cervical  bruit  and  no  history  of  such  symptoms/^  When 
stroke  was  the  only  event  considered,  the  yearly  rate  was  less  than  2%. 

Transient  ischemic  attacks  (TIAs)  pose  another  well  established  risk 
factor  for  stroke.  Surveillance  of  the  population  of  Rochester,  Minnesota 
over  a  fifteen  year  period  documented  an  increased  risk  of  stroke  and  death 
in  people  with  TIAs.^^  Others  have  since  corroborated  this  finding.^®’^'^”'^’  In 
a  twenty  year  retrospective  review  of  the  same  city,  Cartlidge  and  co¬ 
workers  demonstrated  that  the  risk  of  death  at  one  month  after  the  first  TIA 
is  10  times  greater  than  that  in  a  matched  population,  but  drops  thereafter 
to  reach  a  risk  of  1.5  times  that  of  controls  after  two  years.^^  The  risk  of 
stroke  in  the  same  population  was  markedly  elevated  over  the  expected  rate 
in  this  community— 17.5  times  the  expected  rate  at  one  year,  and  9  times  the 
expected  rate  by  the  completion  of  the  study  (6  to  20  years  of  follow-up). 
Like  the  epidemiologic  investigators  of  asymptomatic  bruits,  these 
researchers  found  a  high  rate  of  cardiac  events. 

Also  noteworthy  are  the  findings  in  the  placebo  arm  of  the  Canadian 
Cooperative  Study  Group's  randomized  controlled  trial  of  anti-platelet 
therapy  for  patients  with  a  prior  stroke  or  recent  TIA.^’  This  team  found  a 
cumulative  probability  of  stroke  or  death  of  29%  three  years  after  inclusion 
in  the  study.  This  may  not  represent  a  realistic  estimate  of  the  risk  of 
stroke  after  TIA,  however,  because  the  cohort  included  patients  with  prior 
strokes  in  addition  to  those  with  TIAs.  Although  the  literature  offers  widely 
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varying  estimates  of  stroke  and  death  from  cerebrovascular  disease  due  to 
differences  in  methodology,  it  is  clear  that  asymptomatic  bruits  and 
especially  TIAs  identify  people  at,  increased  risk  for  stroke  and  death. 

Two  other  groups  may  be  at  heightened  stroke  risk  as  well.  First, 
patients  with  prior  strokes  may  be  at  increased  risk  for  subsequent  events.” 
Secondly,  some  argue  that  patients  with  severe  carotid  stenosis  identified 
solely  by  radiographic  studies  are  more  prone  to  have  strokes.” 

Medical  management 

It  is  primarily  in  the  two  major  at-risk  populations—patients  with  carotid 
bruits  and  those  with  TIAs—that  medical  and  surgical  interventions  to  reduce 
strokes  have  been  investigated.  Anti-platelet  therapy  has  become  the 
mainstay  of  medical  protection  in  patients  with  carotid  artery  disease. 

Several  studies  of  prophylactic  aspirin  have  been  conducted,””^’ and  the 
consensus  from  these  trials  is  that  a  25  to  30  percent  reduction  in  the 
incidence  of  stroke  after  transient  ischemic  attacks  can  be  achieved  on  a 
daily  oral  dose  of  1300  mg.^  Examinations  have  also  been  made  of 
supplemental  platelet  anti-aggregants  such  as  sulfinpyrazone”  and 
dipyridamole.”  These  studies  have  found  no  beneficial  effect  of  substituting 
or  adding  a  second  agent  to  daily  aspirin.  Anticoagulation  therapy  is 
generally  shunned  as  prophylaxis  for  at-risk  patients  because  of  its  increased 


5 


incidence  of  hemorrhagic  compications.^*'^®'^  As  mentioned  above,  the  control 
of  other  risk  factors  such  as  hypertension,  diabetes  and  hyperlipidemia  is 
another  important  part  of  stroke  prevention. 

Surgical  management:  carotid  endarterectomy 

The  alternative  strategy  to  manage  the  patient  with  stenotic  carotid 
arteries  is  surgery.  Two  operative  approaches  have  been  attempted,  but  one, 
the  technique  of  extracranial-intracranial  bypass  surgery,  has  been  largely 
rejected  clinically  since  a  large  randomized  trial  failed  to  demonstrate  any 
benefit  of  the  procedure.^^  The  remaining  operation  for  the  prevention  of 
stroke  due  to  carotid  disease  is  the  endarterectomy. 

Carotid  endarterectomy  is  commonly  performed,  costly  and  controversial. 
In  1987,  more  than  80,000  of  these  operations  were  performed  in  the  United 
States,  marking  a  doubling  in  surgical  volume  over  10  years  (Table  1). 

While  the  volume  of  endarterectomy  has  leveled  off  and  even  decreased 
since  the  middle  of  the  1980s,  it  remains  one  of  the  most  common  operation 
performed  in  the  US.^^  In  an  era  of  burgeoning  financial  consciousness  in 
the  health  care  field,  the  greater  than  $1  billion  price  tag  that  these 
operations  carry  collectively^^  cannot  be  ignored. 
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TABLE  1:  CAROTID  ENDARTERECTOMIES  IN  THE  US  BY 


Year 

Operations 

1972 

16,000 

1975 

34,000 

1978 

42,000 

1981 

73,000 

1982 

82,000 

1983 

95,000 

1984 

103,000 

1985 

107,000 

1986 

83,000 

1987 

81,000 

"Estimated  to  the  nearest  thousand.  Includes  endarterectomies 
of  the  head  and  neck  performed  in  non-governmental  acute 
care  hospitals  in  the  United  States. 


Disagreement  over  the  role  of  endarterectomy  exists  because  the 
scientific  evidence  currently  available  is  inconclusive  and  incomplete.  In 
order  to  evaluate  the  operation's  benefit  relative  to  its  risk,  several  factors 
must  be  considered.  First,  the  surgery  must  be  demonstrated  to  reduce  the 
risk  of  stroke.  To  date,  the  results  of  two  randomized  trials  on  this  question 
have  been  published.^'^  The  Joint  Study,^  completed  twenty  years  ago, 
reported  a  9%  reduction  in  aggregate  stroke  and  deaths  for  patients  with 
TIAs  who  received  the  operation— 17.3%  for  surgical  patients  versus  26.2% 
for  the  non-surgical  arm.  The  patients  were  similar  in  the  two  groups  in 
meaningful  prognostic  criteria  such  as  age  and  distribution  of  prior  TIAs. 
Patients  were  followed  for  an  average  of  42  months  post-randomization  and 
their  deficits  were  well  characterized.  The  non-surgical  limb  of  the  trial, 
however,  did  not  receive  aspirin,  the  currently  accepted  medical  alternative. 
Consequently,  the  efficacy  of  surgery  relative  to  its  alternatives  may  not  be 
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as  great  as  the  autors  reported.  Moreover,  The  Joint  Study  considered  the 
long  term  effects  in  the  surgical  limb  only  in  those  patients  who  survived 
their  endarterectomy  without  neurologic  complication.  As  a  result,  a  subset 
of  patients  who  may  have  been  at  greater  risk  of  suffering  strokes  in  the 
future  were  not  observed.  The  other  published  randomized  trial  was 
terminated  prematurely  due  to  the  increased  risk  of  death  observed  in  its 
surgical  arm.“  Surgical  efficacy  from  that  experiment  could  not  be 
determined. 

If  one  assumes  that  carotid  endarterectomy  can  reduce  the  incidence  of 
stroke  due  to  carotid  disease,  the  risk  of  the  procedure  itself  must  be 
weighed  against  that  benefit.  Along  with  the  inherent  dangers  of  any 
surgery-including  hemorrhage,  infection,  cardiac  stress,  and  the  risks  of 
anesthesia— carotid  endarterectomy  carries  the  potential  for  causing  strokes, 
due  either  to  hypoxic  or  embolic  causes.  Several  variations  in  the  standard 
technique  have  been  advocated  to  reduce  the  risk  of  these  untoward  events- 
selective  shunting,  performing  the  procedure  under  regional  anesthesia, 
monitoring  with  electro-encephalography,  and  closing  the  artery  with  a  patch 
graft.  All  of  these  modifications  appear  to  be  capable  of  yielding  low 
complication  rates,  but  none  has  been  shown  to  consistently  outperform  the 
standard  anesthetic,  monitoring  and  surgical  methodology.^’' 

Over  two  hundred  studies  of  the  operative  mortality  and  morbidity  of 
carotid  endarterectomy  have  been  published  during  the  last  fifteen  years, 
reporting  death  plus  stroke  rates  ranging  from  less  than  3%  to  greater  than 


H  Iff: 

ijmmm 


8 


20%  3839  Tremendous  disparity  in  the  quality  of  research  design  appears 
when  reviewing  this  literature,  and  many  factors  contribute  to  the  wide 
discrepancy  in  outcomes  reported.  First,  case  mix  varied  widely,  both 
among  and  within  studies.  Not  only  do  medical  comorbidities  influence 
outcome,  but  also  the  literature  suggests  that  patients  with  tighter  stenoses, 
ulcerated  plaques,  and  prior  neurologic  symptoms  or  evolving  strokes  have 
a  greater  chance  of  complications.”  Few  studies  adequately  describe  their 
study  population  to  allow  for  comparison. 

Differences  in  techniques  of  analysis  and  reporting  of  the  results 
consititute  a  second  major  set  of  reasons  for  the  variations  in  reported 
outcomes.  Because  none  of  the  studies,  with  the  exception  of  the  two  trials 
discussed  above,^”^  randomized  patients,  and  most  of  them  were  performed 
in  a  retrospective  fashion,  issues  of  bias  cannot  be  ignored.  Many  published 
studies  lack  explanation  of  patient  selection  criteria,  descriptions  of  the 
method  and  period  of  follow-up,  and  clear  definition  of  outcomes.* 

Moreover,  the  diagnosis  of  stroke  remains  problematic,  with  much 
interobserver  variability  even  among  trained  neurologists.'^'*’ 

Almost  without  exception,  the  self-reports  of  individual  surgeons  or 
single  institutions^'*^'”'*^  tend  to  have  lower  rates  of  complications  than 
regional  or  multi-center  studies  conducted  by  independent  auditors.'**'*^'**''*’'^ 


*The  response  of  one  major  researcher  in  the  field  when  asked  how  he 
defined  stroke  in  his  study  population  underscores  just  how  unscientific  the 
reporting  can  be:  "A  stroke  is  like  pornography,  you  know  it  when  you  see 

it." 
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This  discrepancy  may  be  due  to  the  multitude  of  potential  biases  associated 
with  reporting  one's  own  data.  Alternatively,  the  difference  in  reported 
outcomes  between  the  single  center  and  multi-center  studies  may  represent 
the  difference  between  the  efficacy  and  the  effectiveness  of  the  procedure: 
that  is  to  say,  the  difference  between  the  operation's  outcomes  under  ideal 
conditions  and  the  realistic  expectations  of  its  performance  under  commonly 
encountered  conditions. 

Publication  selection  biases  may  be  present  as  well.  Only  those  authors 
with  highly  successful  outcomes  are  likely  to  offer  their  results  for  scrutiny. 
In  addition,  journals  are  often  more  willing  to  publish  "positive"  findings 
than  less  successful  results. 

Complications  of  carotid  endarterectomy  other  than  stroke  and  death  are 
rarely  addressed  in  this  vast  literature.  Between  10  and  20%  of  patients 
suffer  cranial  nerve  palsies  of  the  hypoglossal,  glossopharyngeal,  recurrent 
laryngeal,  superior  laryngeal  or  marginal  mandibular  nerve.^^'^^'^  Most 
clinicians  perceive  these  impairments  as  minor  nuisances  relative  to  a  stroke. 

A  small  percentage  of  carotid  arteries  have  been  shown  to  restenose 
after  surgery.  Studies  of  symptomatic  restenoses  reveal  a  wide  spectrum  of 
rates,  ranging  from  1%  to  greater  than  15%. 544536.5738  Differences  in  detection 
techniques  (of  the  symptoms  and  of  the  anatomical  lesions)  and  in  the 
length  and  rigor  of  follow-up  probably  account  for  the  variation  in  findings 
as  much  as  differences  in  surgical  proficiency.  Patients  who  restenose 
derive,  at  best,  temporary  benefit  from  the  operation. 
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Cardiac  events  constitute  a  dangerous  set  of  complications  of  the 
operation.  Because  atherosclerosis  is  a  systemic  disease,  patients  with 
stenotic  carotid  arteries  are  likely  to  have  diseased  coronary  arteries,  too. 
Consequently,  myocardial  infarction  (MI)  occurs  in  a  small  but  not 
insignificant  percentage  of  endarterectomy  patients.  A  retrospective  study  of 
491  patients  found  that  3.3%  of  patients  suffered  an  MI  after  endarterectomy, 
and  that  1%  of  all  patients  died  from  MI  in  the  immediate  post-operative 
period.^^  There  was  a  ten-fold  increase  of  MI  in  the  patients  with  prior 
heart  disease  (defined  as  history  of  angina,  congestive  heart  failure, 
arrhythmia,  prior  MI,  or  use  of  cardiac  medications)  versus  those  without 
such  risk  factors.  The  rates  of  MI  and  fatal  MI  were  identical  to  those 
quoted  above  in  a  separate  study  published  four  years  later.^  The  latter 
work  showed  slightly  less  than  three  times  the  risk  of  MI  in  patients  with 
known  coronary  artery  disease,  either  by  history  or  found  on  EKG  alone,  as 
compared  to  those  lacking  coronary  artery  disease.  Neither  study 
considered  the  rate  of  MI  in  a  matched  cohort  that  did  not  receive  the 
surgery. 

Evaluating  the  options  available  to  the  at-risk  patient 

Clinical  uncertainty  about  the  appropriate  indications  for  the  procedure 
is  reflected  in  the  high  degree  of  regional  variation  in  the  utilization  rates  of 
carotid  endarterectomy.^^'^  For  example,  residents  of  Boston  are  more  than 


Im 

im 


11 


twice  as  likely  to  undergo  carotid  endarterectomy  as  are  residents  of  New 
Haven.^  In  the  absence  of  compelling  results  from  a  prospective 
randomized  study  of  the  operation's  value,  researchers  have  created 
alternative  frameworks  to  evaluate  endarterectomy's  place  in  stroke 
prevention.^'^'“'^^ 

Matchar  and  Pauker  applied  a  Markov  decision  analysis  to  the  question 
of  whether  an  otherwise  healthy  sixty  year  old  with  a  recent  TIA  should 
undergo  carotid  endarterectomy.^  They  found  that  three  factors  could 
strongly  influence  the  choice  of  management:  surgical  risk,  surgical  efficacy 
and  the  natural  history  of  the  disease  as  reflected  in  stroke  rate  without 
surgery.  By  looking  at  each  of  these  key  variables  at  the  extremes  of  their 
plausible  ranges  (derived  from  a  critical  review  of  the  literature),  they 
created  multiple  permutations  of  their  decision  model.  Only  when  the 
model  was  weighted  to  prefer  surgery-based  upon  the  high  risk  of  stroke 
without  endarterectomy  and  the  most  optimistic  estimate  of  surgical 
efficacy— could  a  benefit  (of  four  additional  months  of  "quality  adjusted"  life 
expectancy)  be  shown  for  low  risk  surgery  (defined  therein  as  a  1% 
mortality  and  4%  non-fatal  stroke  rate).  The  balance  tipped  in  favor  of 
medical  management  when  their  high  estimate  of  surgical  risk  (4%  mortality, 
8%  non-fatal  stroke)  was  substituted.  No  threshold  analysis’  of  surgical  risk 
was  included  in  their  publication.  When  the  model  was  altered  to  assume 


’The  value  at  which  the  computed  benefits  of  both  treatment  options  are 
equal. 
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low  surgical  efficacy  (20%  of  strokes  prevented,  instead  of  50%)  or  to 
include  a  risk  of  stroke  without  surgery  of  3%  per  year  (instead  of  5%),  as 
is  estimated  for  asymptomatic  patients, the  relative  benefits  of  either 
therapy  were  negligible  (a  "toss-up")  or  medical  management  was  clearly 
preferable  to  surgery. 

Possible  risk  factors  for  complications  in  carotid  endarterectomy 

Matchar  and  Pauker's  analysis  highlights  the  importance  of  the  surgical 
complication  rate  in  deciding  whether  or  not  to  operate.  Clearly,  that 
surgical  risk  varies  both  among  patients  and  across  providers  (surgeons  and 
institutions).  It  is  not  surprising,  therefore,  that  many  clinicians  argue  that 
carotid  endarterectomy  should  be  restricted  to  specialized  centers  or  to  those 
with  documented  operative  stroke  plus  death  rates  of  5  or  6%.”'“'^ 

A  decade  ago  Luft  and  colleagues  suggested  that  major  surgical 
procedures  be  regionalized  to  "centers  of  excellence."  The  argument  is  based 
on  the  assumption  that  outcomes  are  better  where  a  procedure  is  performed 
more  frequently;  empirical  evidence  supports  this  notion.^”  Observed 
relationships  between  volume  and  outcome  may  be  due  to  honing  skills  but 
may  also  represent  patient  selection  biases.  The  volume-outcome 
relationship  has  been  explored  for  carotid  endarterectomy  in  several  studies. 
Two  studies  found  no  correlation  between  hospital  size  and  outcomes,^^'^ 
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whereas  a  third  investigation  identified  lower  complication  rates  in  hospitals 
with  greater  than  700  beds  as  compared  to  smaller  institutions.'^ 

The  risk  of  all  surgery  is  generally  considered  to  rise  with  age. 
Specifically  for  endarterectomy,  however,  three  studies  found  no  significant 
increased  risk  in  patients  over  the  age  of  75  or  80,  compared  either  to  a 
younger  cohort  or  to  a  matched  elderly  group  managed  non-surgically,  and 
advocated  the  use  of  the  procedure  in  elderly  patients.^^'^'*'^ 

The  use  of  claims  data  to  study  carotid  endarterectomy 

With  the  introduction  of  easily  accessible,  powerful  computers  during  the 
last  decade,  large  data  bases  of  clinical  information  were  assembled  and 
their  analytic  manipulation  became  common.  At  the  same  time  that 
researchers  were  discovering  the  use  of  data  sets  to  evaluate  the  clinical 
practice  of  medicine,  policy-makers  were  becoming  increasingly  concerned 
about  the  cost  of  health  care.  With  interests  in  both  cost  and  quality,  the 
attention  of  the  Health  Care  Financing  Administration  (HCFA),  the  federal 
agency  responsible  for  managing  the  Medicare  program,  has  turned  toward 
methods  to  review  the  care  that  it  funds.^*  Not  only  does  the  HCFA  have 
an  interest  in  using  large  information  data  bases  to  evaluate  medical  care, 
the  agency  has  the  data  itself  to  probe  these  issues. 

Whenever  a  Medicare-eligible  patient  is  discharged  from  any  hospital  in 
the  US  (except  a  Veteran's  Affairs  hospital),  the  HCFA  is  billed  using  a 
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standardized  claims  form  that  records,  among  other  things,  the  patient's  age, 
principle  diagnoses  (up  to  five),  principle  procedures  (up  to  3),  dates  of 
admission,  discharge  and  first  procedure,  and  discharge  destination.  The 
diagnoses  and  procedures  are  coded  according  to  a  standardized 
classification  scheme,  the  International  Classification  of  Diseases,  Ninth 
Revision,  Clinical  Modification  (ICD-9-CM).^  These  Medicare  billing  records 
are  stored  in  the  HCFA  Medicare  Provider  Analysis  and  Review  (MEDPAR) 
file. 

A  separate  set  of  data,  the  Health  Insurance  Skeletonized  Eligibility 
Write-off  (HISKEW)  file,  maintains  information  on  whether  a  beneficiary  has 
died  and  is  therefore  no  longer  eligible  for  Social  Security  benefits.  Each 
death  is  dated  and  recorded  regardless  of  whether  the  person  died  in  a 
hospital.  A  third  set  of  data  maintained  by  the  HCFA,  the  Provider  of 
Services  (POS)  file,  describes  characteristics  of  all  institutions  that  offer 
services  to  Medicare  beneficiaries. 

All  the  files  listed  above  may  be  linked  for  analytic  purposes,  and 
specific  records  may  be  concatenated  to  create  chronological  expressions  of  a 
person's  in-hospital  care.  An  early  example  of  the  use  of  this  technique  is 
the  study  performed  by  Wennberg,  et.al.,  to  examine  complications  following 
prostatectomy.^® 

In  the  study  reported  here,  the  Medicare  claims  files  for  the  six  New 
England  states  were  used  to  consider  the  effects  of  patient  age  and  surgical 
volume  (by  institution)  on  the  complication  rate  of  carotid  endarterectomy. 
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The  question  of  whether  such  information  could  be  used  to  help  evaluate 
the  performance  of  carotid  endarterectomy  at  individual  hospitals  was  also 
explored. 

The  Medicare  claims  files  are  a  useful  tool  to  investigate  these  issues 
because  they  allow  for  access  to  a  large  series  of  cases  performed  across  an 
entire  region,  because  they  provide  excellent  follow-up  beyond  the  discharge 
date  of  the  initial  hospitalization,  and  because  they  report  on  the  majority  of 
patients  who  receive  the  operation.  Seventy  percent  of  the  patients  receiving 
carotid  endarterectomy  are  65  years  of  age  or  older.^ 
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Methods 

Data  source.  Data  for  the  current  study  were  obtained  from  the  MEDPAR, 
HISKEW,  and  POS  files  maintained  by  the  HCFA. 

Selection  of  the  Study  Population.  All  2171  hospital  discharge  abstracts 
submitted  for  New  England  resident  Medicare  beneficiaries  65  years  of  age 
and  older  discharged  between  April  1,  1984  and  June  30,  1985  who 
underwent  carotid  endarterectomy  (ICD-9-CM  code  number  38.12)  were 
reviewed.  After  eliminating  the  81  patients  who  received  surgery  outside 
New  England  and  1  patient  with  inconsistent  coding,  2,089  patients  were 
selected  for  inclusion  in  the  study. 

Observation  Period.  The  post-operative  period  was  defined  as  thirty  days 
from  the  date  of  endarterectomy  In  the  MEDPAR  file  at  the  time  of  this 
study,  only  the  primary  procedure  was  dated.  If  the  endarterectomy  was  not 
listed  as  the  primary  procedure  (n  =  83,  3.8%  of  cases),  the  date  of  surgery 
for  the  index  endarterectomy*  was  assigned  to  the  midpoint  of  the  hospital 


‘"Index  endarterectomy"  denotes  the  first  endarterectomy,  regardless  of 
how  many  of  the  procedures  the  patient  received.  The  "index 
hospitalization"  refers  to  the  admission  during  which  the  index 
endarterectomy  was  performed. 
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stay.  (All  reported  analyses  were  repeated  using  the  date  of  admission  as 
the  date  of  surgery  for  these  83  cases;  none  of  the  analyses  were  affected.) 

If  a  patient  received  more  than  one  endarterectomy,  either  on  the  same 
artery  or  the  contralateral  side,  and  regardless  of  whether  they  were 
performed  during  the  same  hospitalization,  all  analyses  were  performed 
assuming  the  post-operative  period  included  the  30  days  after  the  first 
procedure. 

Outcomes.  Deaths  were  determined  by  searching  the  HISKEW  file.  Stroke 
was  ascertained  by  inference  from  the  diagnostic  codes  recorded  on  the 
discharge  abstracts  in  the  MEDPAR  file. 

Two  sets  of  problems  arose  in  identifying  post-operative  strokes.  Some 
codes  describe  conditions  that  may  be  present  in  victims  of  strokes  of  the 
carotid  distribution,  but  also  are  found  in  other  conditions  (eg.  ataxia).  Only 
those  codes  that  were  considered  hallmark  sequelae  of  carotid  strokes  were 
included  in  the  codes  identifying  stroke  (see  Appendix  A).  Second,  because 
diagnosis  codes  are  neither  entered  onto  the  abstraction  forms  in 
chronological  order  nor  dated,  in  several  cases  ambiguity  existed  over  the 
temporal  relationship  of  the  stroke  to  the  surgery.  Timing  the  stroke  is 
problematic  because  in  some  cases  the  decision  to  operate  may  be  based 
upon  a  prior  stroke. 

To  address  this  problem,  I  developed  two  algorithms  (described  in 
Appendix  B)  to  find  strokes  in  patients  receiving  carotid  endarterectomy. 


.4 
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The  "definite  stroke"  algorithm  identified  patients  whose  stroke  could  only 
have  occurred  during  the  thirty  days  following  the  surgery  (n  =  35).  The 
"possible  stroke"  algorithm  also  included  patients  whose  event  could  not  be 
placed  with  certainty  in  relation  to  the  operation  (n  =  161).  In  other  words, 
the  first  method  chose  patients  either  with  an  ICD-9-CM  code  for  a 
neurologic  surgical  complication  during  the  index  admission  or  with  a  code 
consistent  with  stroke  on  readmission  within  thirty  days.  The  second 
selection  technique  included  all  patients  identified  in  the  first  algorithm  plus 
those  with  ICD-9-CM  codes  consistent  with  stroke  during  the  index 
admission. 

Based  on  these  definitions,  I  calculated  the  proportion  of  patients  who 
experienced  the  following  outcomes  within  30  days  of  surgery:  death,  death 
or  definite  post-operative  stroke  (the  minimum  estimate  of  the  stroke  plus 
death  risk),  and  death  or  any  possible  post-operative  stroke  (the  maximum 
risk  of  death  plus  stroke  that  can  be  ascertained  from  the  discharge 
abstract).  Patients  suffering  fatal  strokes  were  counted  only  once  in  the 
combined  stroke  and  death  totals.  Confidence  intervals  for  these  proportions 
were  calculated  according  to  the  exact  method.^^ 

Independent  Variables.  Patients'  age,  sex  and  race  were  ascertained  from 
the  discharge  records  in  the  MEDPAR  file.  To  identify  important  medical 
conditions  of  patients,  diagnostic  codes  were  examined  from  both  the 
surgical  (index)  admission  and  from  any  admissions  whose  discharge  date 
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fell  within  the  6  months  preceding  the  date  of  the  index  surgical  procedure, 
in  the  method  previously  described  by  Wennberg/*  If  an  index  admission 
diagnosis  could  reasonably  be  interpreted  as  a  potential  complication  of  the 
endarterectomy,  a  patient  was  coded  as  having  that  condition  only  if  it  v/as 
also  present  on  an  admission  preceding  the  index  hospitalization.  The 
following  conditions  were  evaluated  as  risk  factors  for  adverse  outcomes: 
coronary  artery  disease,  valvular  heart  disease,  congestive  heart  failure, 
stroke,  pulmonary  disease,  renal  disease,  diabetes,  and  the  presence  of  any 
hospital  admission  during  the  previous  6  months.  The  ICD-9-CM  codes 
used  to  define  these  conditions  are  provided  in  Appendix  C. 

Hospital  surgical  volume  was  defined  based  on  the  number  of  index 
carotid  endarterectomies  performed  by  each  hospital  on  the  Medicare 
population  during  the  15  month  study  period.  Institutional  size  was 
defined  as  a  continuous  variable  based  on  the  number  of  hospital  beds. 
Hospital  teaching  status  was  defined  as  a  continuous  variable  based  on  the 
number  of  housestaff  in  accredited  residency  programs  per  hospital  bed. 

Analysis.  Multiple  logistic  regression  was  used  to  test  the  association 
between  each  of  the  three  outcomes  and  the  clinical  and  institutional 
predictors  described  above.®”  Of  the  clinical  variables  considered,  only  the 
presence  of  coronary  artery  disease  was  significantly  associated  with  30  day 
mortality  at  the  .10  level  after  controlling  for  age,  sex,  and  surgical  volume. 
Both  sex  and  coronary  artery  disease  were  included  in  the  reported  logistic 
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regression  models.  Alternative  specifications  of  the  model  in  which  teaching 
status  and  hospital  size  were  treated  as  both  dichotomous  and  ordinal 
variables  were  examined.  Regardless  of  the  specification  of  the  model,  the 
results  were  similar  to  those  presented  below.  Two-way  interactions 
between  surgical  volume  and  age,  and  age  and  sex  were  examined  and  were 
not  significant. 

The  binomial  distribution  was  used  to  test  the  hypothesis  that  the 
observed  number  of  deaths  plus  definite  strokes  at  an  individual  hospital 
exceeded  the  number  that  would  have  been  expected  had  the  hospital's 
outcomes  equalled  those  of  the  highest  volume  hospitals. 
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Results 

Operative  mortality.  Operative  mortality  increased  markedly  with  age 
(Table  2).  Those  over  age  70  faced  a  progressively  increased  risk,  with  those 
over  age  80  facing  a  more  than  four-fold  increase  in  the  adjusted  odds  of 
death  compared  to  those  ages  65  to  69. 

Operative  risk  was  also  higher  at  hospitals  performing  few 
endarterectomies  on  the  Medicare  population  (Table  3).  Patients  undergoing 
carotid  endarterectomy  in  hospitals  in  each  of  the  three  lower  volume 
groups  faced  an  increased  risk  of  death  compared  to  those  receiving  surgery 
in  hospitals  performing  more  than  40  endarterectomies  per  year.  The 
increased  risk  associated  with  low-volume  surgery  was  consistent  across  all 
specifications  of  the  model  examined.  When  a  dichotomous  variable  was 
substituted  for  the  three  levels  of  lower  hospital  volume  in  an  otherwise 
identical  model,  the  adjusted  odds  ratio  for  30  day  mortality  at  hospitals 
performing  40  or  fewer  endarterectomies  per  year  compared  to  higher 
volume  hospitals  was  2.83  (95%  Cl  1.11,  7.19).  When  volume  is  treated  as  a 
continuous  variable  in  the  same  model  the  conclusion  is  still  statistically 
significant  (p  =  .03),  and  remains  so  after  controlling  for  teaching  status  and 
hospital  size. 

Risk  of  death  or  stroke.  Because  it  was  not  possible  to  characterize  the 
timing  in  relation  to  surgery  of  the  majority  of  strokes  recorded  on  the 
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discharge  abstract,  both  a  minimum  and  a  maximum  estimate  of  post¬ 
operative  death  plus  stroke  risk  are  presented  in  Tables  4  and  5.  Regardless 
of  the  definition  used,  the  risk  of  death  or  stroke  increases  significantly  with 
age  and  at  lower  volume  hospitals. 

Individual  hospital  outcomes.  All  hospitals  were  identified  where  the 
number  of  observed  deaths  or  definite  strokes  was  greater  than  would  have 
been  expected  (p  <  .10  by  chi-square  analysis),  using  as  the  standard  of 
comparison  the  outcome  rate  in  those  hospitals  performing  more  than  40 
carotid  endarterectomies  per  year  (Table  6).  Twenty-two  of  the  139  New 
England  hospitals  performing  endarterectomy  were  found  to  have  high 
complication  rates  compared  to  high  volume  hospitals.  Only  7  New 
England  hospitals  performed  more  than  40  carotid  endarterectomies  per  year 
on  the  Medicare  population,  while  50  hospitals  performed  5  or  fewer. 

The  complication  rates  of  the  7  high  volume  hospitals  all  fall  in  the  low 
range  (Table  7).  No  institution  in  that  group  had  a  death  plus  definite 
stroke  rate  greater  than  the  average  for  hospitals  in  any  other  range  of 


volume. 
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Discussion 

This  study  identifies  two  risk  factors  for  complications  from  carotid 
endarterectomy.  First,  the  results  demonstrate  a  markedly  elevated  mortality 
risk  with  increasing  age.  Two  thirds  of  the  New  England  Medicare  patients 
undergoing  endarterectomy  were  over  the  age  of  70  and  faced  an  average 
three-fold  increased  risk  of  operative  death  compared  to  younger  patients. 
The  non-fatal  stroke  trend  parallels  the  mortality  findings.  These  results 
contrast  with  the  work  of  other  authors  who  have  found  no  significant 
increase  in  operative  mortality  and  morbidity  in  elderly  patients  undergoing 
carotid  endarterectomy.^”'”'’'^ 

Ouriel  and  colleagues  retrospectively  compared  the  outcomes  of  470 
carotid  endarterectomies  at  their  own  institution.”  Seventy  seven  of  the 
cases  were  performed  on  patients  older  than  75  years  of  age.  The 
distribution  of  surgical  indications  between  the  two  groups  was  similar, 
although  other  clinical  comorbidities  were  not  considered.  None  of  the 
elderly  patients  died  post-operatively  compared  to  two  (0.5%)  fatalities  in  the 
younger  cohort.  The  aggregate  stroke  and  death  rates  were  3.9%  in  the 
elderly  and  3.6%  in  those  75  year  of  age  and  younger  (difference  not 
statistically  significant).  In  contrast  to  these  results,  I  found  a  case  fatality 
rate  of  3.6%  in  the  685  patients  75  years  and  older.  Several  differences  in 
methodology  may  explain  the  variation  in  the  two  sets  of  mortality  rates. 
First,  Ouriel  is  not  explicit  in  the  length  of  the  post-operative  period  studied 
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and  he  is  likely  to  have  used  the  index  hospitalization  as  his  observation 
period.  I  observed  all  patients  for  30  days  regardless  of  the  site  of 
convalescence.  In  addition,  over  25%  of  the  patients  in  Ouriel's  series  were 
operated  on  for  "non-hemispheric  symptoms,"  a  dubious  indication  for 
carotid  endarterectomy.  It  is  unknown  what  percentage  of  the  New  England 
cohort  received  the  procedure  for  non-focal  complaints.  The  most  important 
distinction  between  the  two  studies  is  that  Ouriel's  patients  were  all 
operated  on  at  one  hospital,  and  thus  are  not  equivalent  to  the  community¬ 
wide  experience  represented  by  the  New  England  cohort. 

Making  use  of  6000  endarterectomy  patient  records  in  the  Cleveland 
Vascular  Society  registry,  Plecha  and  co-workers  also  reported  no  significant 
elevation  in  the  stroke  and  death  rates  of  patients  75  years  of  age  and 
older.^'*  While  they  concluded  that  "carotid  endarterectomy  is  tolerated  well 
by  patients  at  all  age  levels,"  their  data  suggest  an  alternative  trend.  The 
mortality  rate  was  1.5%  in  patients  less  than  75  years  of  age,  2.2%  in  those 
75  to  79,  and  2.7%  in  those  patients  80  years  and  older  (difference  not 
statistically  significant  at  the  .05  level).  It  is  interesting  to  note  that  the 
death  rates  for  my  New  England  cohort  were  1.9%,  3.2%,  and  4.7%  in  those 
same  age  ranges  (see  Table  2).  While  the  pattern  is  the  same,  the  mortality 
rate  is  consistently  higher  in  the  New  England  population.  This  variation 
can  be  explained  by  the  differences  in  the  two  studies'  surgeons  and  the 
accounting  methods  employed  to  calculate  death  rates.  Unlike  my  study 
which  includes  all  endarterectomies  on  patients  65  years  of  age  and  older. 
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Plecha  draws  only  from  the  experiences  of  a  regional  specialty  society  that 
enumerates  strict  membership  criteria  and  scrutinizes  its  participants  results.®’ 
Consequently,  only  results  from  veteran  and  highly  credentialed  surgeons 
who  are  willing  to  let  their  colleagues  review  their  outcomes  appear  in 
Plecha's  report.  Another  reason  for  lower  death  rates  in  the  Cleveland 
Vascular  Society's  registry  is  that  they  consider  the  post-operative  period  to 
last  only  as  long  as  the  index  hospitalization,  not  a  full  30  days. 

Furthermore,  they  identified  cases  by  procedures,  not  by  patients,  so  that 
persons  receiving  serial  bilateral  endarterectomies  or  re-operations  were 
counted  multiple  times  in  their  denominator.  These  methodologic 
differences  all  lead  to  lower  reported  mortality  rates,  partially  obscuring  the 
risk  of  the  operation  in  elderly  patients  over  an  entire  region. 

Similar  to  Ouriel  and  to  Plecha,  Richardson  and  Main  "found  no 
tendency  toward  increased  negative  outcomes  as  a  result  of  the  patients' 
advancing  age."®”  Their  study  made  use  of  all  705  endarterectomies 
performed  in  41  Kentucky  hospitals  over  an  18  month  period.  They  clearly 
reported  their  detection  and  follow-up  techniques;  however,  they  did  not 
publish  sufficient  data  for  the  above  conclusion  to  be  scrutinized. 

A  second  conclusion  of  my  research  is  the  inverse  relationship  between 
institutional  surgical  volume  and  complication  rates.  Patients  undergoing 
surgery  at  low-volume  hospitals  faced  a  three-fold  increased  risk  of  death 
compared  to  those  receiving  surgery  at  high-volume  hospitals.  On  this 
issue,  too,  there  is  divergence  from  the  conclusions  of  others.®”'^’^ 
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Kempczinski  and  colleagues  compared  hospital  volume  and  outcomes  in 
16  Cincinnati  hospitals,  stratifying  by  annual  volumes  of  less  than  50,  50  to 
100,  and  greater  than  100  endarterectomies.^  They  found  no  significant 
difference  in  mortality  or  stroke  morbidity  among  the  groups.  While  the 
overall  mortality  rates  are  comparable  in  their  study  and  mine  (2.3%  and 
2.5%,  respectively),  my  results  identify  a  clear  pattern  of  increased  risk  with 
diminishing  surgical  volume.  Kempczinski  and  colleagues  stratified  at  much 
larger  volumes  than  I  chose,  and  thus  may  have  obscured  differences  at 
hospitals  performing  relatively  few  endarterectomies.  The  difference  may 
also  reflect  a  more  uniform  standard  of  care  among  the  16  Cincinnati 
hospitals  than  I  found  across  139  institutions  distributed  over  six  states. 

Using  their  study  of  41  Kentucky  hospitals,  Richardson  and  Main  could 
"not  establish  a  link  between  total  hospital  volume  of  [carotid 
endarterectomy]  and/or  vascular  operations  of  other  types  and  untoward 
outcomes. Like  their  assertion  for  the  link  between  age  and  outcome,  they 
offer  no  data  with  which  to  evaluate  this  conclusion.  Similarly,  Hertzer 
reports  that  in  the  Cleveland  Vascular  Society  registry  data  set  "hospital  size 
had  no  measurable  effect  on  the  mortality  or  stroke  morbidity  rate  of  carotid 
endarterectomy"  without  showing  the  figures  to  support  his  position.^^ 

There  are  several  limitations  to  my  study^s  methods  that  must  be 
recognized.  One  potential  criticism  is  that  the  data  used  to  identify  strokes 
are  imperfect.  The  clinical  information  found  in  Medicare  files  is  gleaned 
from  forms  designed  and  submitted  for  billing  purposes.  The  diagnosis  and 
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procedure  information  is  recorded  on  the  discharge  summary  by  medical 
personnel,  and  then  is  converted  to  ICD-9-CM  codes  by  hospital  abstractors. 
After  passing  through  fiscal  intermediaries,  claims  files  are  subsequently 
submitted  to  the  HCFA  where  they  are  stored  on  computers  in  the 
MEDPAR  file.  Clinical  details  may  be  lost  or  misinterpreted  in  the 
collection  or  storage  process.  In  addition,  skeptics  point  out  that  there  is 
clear  financial  incentive  for  hospitals  to  bias  coding  toward  larger 
remunerations,  especially  since  the  implementation  of  diagnosis-related 
groups  (DRGs)  as  the  primary  reimbursement  mechanism  for  Medicare.” 

The  accuracy  of  the  clinical  information  has  not  been  well  studied,  but  there 
is  evidence  that  biases  in  the  reporting  of  comorbidities  have  emerged  since 
the  DRG  system  was  inplemented  in  1983.”  Moreover,  even  if  the  data  is 
gathered  appropriately,  the  claims  forms  offer  no  means  to  grade  the 
severity  of  clinical  symptoms.  For  stroke  victims,  the  degree  of  symptoms 
and  functional  status  are  important  for  evaluating  outcome. 

The  difficulty  of  timing  strokes  due  to  limited  information  in  the 
MEDPAR  claims  file  poses  a  second  problem  with  the  study's  methodology. 
As  discussed  above,  based  on  the  claims  data  it  was  not  always  possible  to 
know  whether  a  stroke  was  the  indication  for,  or  a  complication  of,  the 
surgery.  In  an  attempt  to  handle  this  ambiguity,  two  algorithms  were 
developed  to  select  a  lower  and  upper  limit  of  the  number  of  possible  post¬ 
operative  strokes  suffered.  The  true  stroke  rate  falls  between  these  two 
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values,  but  the  exact  value  cannot  be  ascertained  with  the  current  method  of 
data  collection. 

Two  lines  of  evidence  suggest  that  the  actual  risk  is  considerably  higher 
than  the  4.2%  minimum  estimate  of  death  plus  stroke  risk  presented  here 
and  may  even  be  closer  to  the  maximum  estimate  of  11.9%.  First,  a 
population-based  sample  of  Medicare  patients  undergoing  carotid 
endarterectomy  in  1981  were  found  to  have  a  combined  death  plus  stroke 
risk  of  9.8%  based  on  review  of  hospital  records.^  Second,  each  of  six 
recently  reported  case  series  that  described  the  outcomes  of  carotid 
endarterectomy  across  multiple  hospitals  reported  a  ratio  of  non-fatal  strokes 
to  deaths  of  approximately  two-to-one.'**^'^'^'®^  Applying  this  ratio  to  the 
data  presented  here  would  suggest  a  stroke  plus  death  risk  of  7.5%  overall. 

In  order  to  address  these  coding  and  timing  issues  associated  with  post¬ 
operative  stroke  identification,  I  performed  a  pilot  study  of  the  accuracy  of 
the  claims  data  (see  Appendix  D).  I  compared  strokes  recognized  by  the 
claims  data  (using  both  algorithms)  to  strokes  found  by  chart  review  in  a 
sample  of  patients.  The  conservative  algorithm  was  found  to  be  50% 
sensitive  (confidence  interval:  15%  to  85%)  and  96%  specific  (Cl:  90%  to 
99%)  for  identifying  complications.  The  liberal  method,  used  to  find  all 
strokes  that  may  have  occurred  post-operatively,  was  88%  sensitive  (Cl:  65% 
to  99%)  and  80%  specific  (Cl:  68%  to  91%).  Its  positive  predictive  value  was 
41%  (Cl:  18%  to  64%).  The  confidence  intervals  for  these  values  are  large. 


This  effort  represents  a  preliminary  study  of  small  sample  size  that  will 
need  larger  follow-up. 
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A  similar  examination  of  the  validity  of  Medicare  claims  data  has 
recently  been  undertaken  on  a  much  larger  scale  by  Fisher.*^  He  compared 
the  data  from  the  National  DRG  Validation  Study  with  the  results  of  chart 
reabstractions  performed  by  reviewers  from  Health  Data  Institute.  He  was 
not  specifically  interested  in  the  temporal  relationship  of  diagnoses  and 
procedures.  For  a  similar,  although  not  identical,  definition  of  stroke  by 
ICD-9-CM  codes,  the  Medicare  claims  data  were  found  to  be  92%  sensitive 
(Cl:  88%  to  95%),  97%  specific  (Cl:  96%  to  97%),  and  to  have  a  positive 
predictive  value  of  71%  (Cl:  65%  to  76%).  Interestingly,  only  90%  (Cl:  73% 
to  98%)  of  carotid  endarterectomies  were  identified  by  the  claims  data. 

Critics  may  also  question  the  quality  of  the  mortality  figures.  This 
information  is  collected  by  the  Social  Security  Administration  (SSA)  for  the 
purpose  of  controlling  benefits  paid  by  the  federal  government.  A  recent 
internal  SSA  study  of  the  ascertainment  of  death  found  the  reporting  to  be 
highly  accurate.  Prior  to  1982,  the  agency  used  data  that  was  evaluated  to 
be  97%  sensitive  for  finding  deaths,  when  compared  to  state  vital  statistics 
(their  "gold  standard").  Since  that  time,  the  even  more  accurate  data  from 
the  states'  vital  statistics  has  been  encorporated  into  the  SSA  death  fhes.®* 

The  use  of  an  independent  data  gatherer  (i.e.  the  SSA)  to  find  fatalities 
offers  an  advantage  over  all  other  studies  of  the  complications  of 
endarterectomy.  While  patients  who  have  died  after  discharge  but  within 
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the  post-operative  period  (defined  herein  as  30  days)  may  be  lost  in  some 
studies,  there  is  complete  follow-up  with  this  method.  The  same  may  be 
assumed  about  the  completeness  of  follow-up  for  strokes  after  discharge, 
because  all  rehospitalizations  are  present  in  the  MEDPAR  file,  regardless  of 
whether  readmissions  are  to  the  same  institutions  or  to  any  other  hospitals 
in  New  England. 

A  separate  set  of  criticisms  raised  about  the  methodology  relates  to  the 
study  population.  Like  many  other  reports  on  the  outcomes  of 
endarterectomy  surgery,  the  operative  candidates  here  were  not  a  uniform 
group  with  a  single  risk  for  stroke  or  death.  While  comorbidities  were 
considered  in  the  analysis,  the  indication  for  surgery  in  these  patients  could 
not  be  ascertained  from  the  data  set.  The  risk  of  surgical  complications 
appears  to  be  lowest  in  asymptomatic  candidates,  intermediate  in  those  with 
TIAs,  and  greatest  in  patients  with  strokes  in  evolution  or  completed 
strokes.^^  In  addition,  some  complications,  especially  cardiac  events,  may 
have  been  the  consequence  of  another  aspect  of  patient  care.  Some  surgeons 
advocate  prophylactic  endarterectomy  prior  to  coronary  artery  bypass 
grafting  (CABG)  in  those  patients  with  stenotic  carotid  arteries.  In  instances 
when  both  operations  are  performed  during  the  same  hospitalization  cardiac 
events  are  more  likely  due  to  CABG  than  to  endarterectomy.  Thirty  patients 
in  this  study  cohort  had  both  procedures  performed  during  the  index 
admission,  and  there  were  no  deaths  in  the  follow-up  period. 
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Finally,  some  have  argued  that  the  diseases  coded  in  discharge  abstracts 
may  not  provide  sufficient  information  to  adequately  control  for  severity  of 
illness  and  comorbidity.®^  The  technique  employed  to  control  for  case  mix 
identified  coronary  artery  disease  as  the  only  medical  comorbidity  to  act  as 
an  independent  risk  factor  for  death.  This  conclusion  is  consistent  with  the 
large  amount  of  evidence  that  more  patients  die  of  coronary  than  carotid 
artery  disease  immediately  after  endarterectomy.  The  claims  data  allowed 
for  only  a  dichotomous  assessment  of  coronary  artery  disease  (i.e.  present 
versus  absent),  however,  as  opposed  to  more  subtle  characterizations  of  that 
condition  such  as  the  Goldman  index.®®'®’ 

There  may  be  differences  in  case  mix  between  hospitals  that  perform 
many  and  few  endarterectomies.  Such  a  bias  could  potentially  misrepresent 
the  volume-outcome  relationship  in  either  direction.  If  high  volume 
hospitals  operate  on  more  patients  who  have  a  low  risk  for  stroke  without 
surgery  (more  asymptomatic  patients,  for  example)  this  could  explain  both 
their  high  endarterectomy  volumes  and  their  low  complication  rates.  Also 
plausible,  though,  is  the  scenario  that  high  volume  hospitals  serve  as  referral 
centers  that  receive  more,  not  less,  challenging  cases.  Furthermore,  if  the 
complication  rates  are  not  due  to  operative  efficacy,  the  findings  may 
indicate  that  high  volume  hospitals  are  better  able  to  choose  appropriate 
surgical  candidates.  Because  comorbidities  are  imperfectly  addressed  by  the 
claims  data,  questions  remain  on  this  issue. 
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While  results  from  randomized  controlled  trials  of  the  surgery  would  be 
preferable  to  all  other  means  of  data  collection  for  assessing  outcomes,  that 
information  is  not  currently  available.  The  only  completed  trial  was 
performed  over  twenty  years  ago,“  and  its  results  may  no  longer  be  valid 
because  of  improved  surgical  techniques  and  post-operative  management. 
Several  large  randomized  controlled  trials— both  for  asymptomatic  patients’®'®^ 
and  those  with  TIAs^^— are  ongoing,  but  most  of  their  results  will  not  be 
known  until  at  least  the  middle  of  the  1990s.  A  preliminary  report  from  the 
asymptomatic  arm  of  the  Veteran's  Administration  cooperative  study 
revealed  a  1.9%  risk  of  death  and  a  2.4%  risk  of  non-fatal  stroke  at  thirty 
days,  but  no  independent  review  or  formal  publication  of  these  data  has  yet 
occurred.”  In  the  interim,  strong  disagreement  remains  over  which 
candidates,  if  any,  are  appropriate  for  surgery. 

Even  when  the  results  of  the  current  randomized  controlled  trials  are 
known,  uncertainty  will  remain.  These  studies  will  provide  data  on  a  best 
case  scenario,  but  not  on  the  spectrum  of  surgery  that  is  presently  available. 
All  of  these  trials  are  being  performed  at  major  academic  institutions,  where 
surgical  volumes  are  high,  and  thus  outcomes  are  presumably  better  than 
average.  Also,  one  of  the  trials  (the  ACAS  study)  requires  that  a  surgeon 
perform  a  minimum  number  of  endarterectomies  before  being  allowed  to 
operate  on  study  patients,  and  that  an  institutional  audit  be  performed  if  a 
center  exceeds  a  threshold  complication  rate  during  the  trial.” 
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Given  the  current  uncertainty  about  the  value  of  carotid  endarterectomy, 
the  importance  of  operative  risk  relative  to  the  expected  benefit  of  the 
procedure,  and  the  wide  variation  in  post-operative  complication  rates  at 
individual  hospitals,  patients  and  their  physicians  should  seek  reliable  data 
on  the  operative  risk  of  death  and  stroke  at  the  hospital  where  the  operation 
is  to  be  performed.  These  data  are  currently  unavailable.  One  solution 
would  be  for  individual  hospitals  to  review  their  records  and  report  their 
results.  Such  a  monitoring  program  would  be  limited  by  the  degree  to 
which  hospitals  could  ascertain  post-discharge  events  and  by  concern  about 
the  potential  conflict  of  interest  for  hospitals  collecting  and  reporting  the 
data. 

This  study  suggests  a  practical  alternative  for  reviewing  and 
disseminating  the  data  for  the  70%  of  patients  undergoing  endarterectomy 
who  are  over  the  age  of  65^  and  thus  eligible  for  Medicare.  Data  in  the 
claims  files  could  be  used  to  establish  an  ongoing  monitoring  program  that 
would  identify  all  patients  undergoing  endarterectomy  and  determine  post¬ 
operative  mortality.  Chart  reviews  could  be  carried  out  to  identify  post¬ 
operative  stroke  rates  and  accurately  classify  patients'  severity  of  illness  and 
comorbidity.  Accurate  risk  stratification,  whether  by  age  or  clinical 
indications  or  both,  would  improve  hospital  comparisons  and  enhance 
individual  clinical  decision  making.  The  effort  required  to  obtain  the 
necessary  supplementary  information  through  focused  chart  review  would  be 
far  less  than  that  required  to  establish  a  prospective  data  collection  system. 
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The  data  could  be  made  available  to  physicians  and  patients  to  provide 
them  with  the  best  available  information  with  which  to  decide  on  the 
management  of  each  unique  case.” 

An  alternative  approach  for  use  of  the  claims  data  would  rely  on  the 
liberal  algorithm  to  identify  all  possible  strokes  in  the  Medicare  population. 
This  would  act  as  a  screen  to  find  the  small  share  of  charts  in  need  of  chart 
review  so  that  only  the  true  post-operative  strokes  would  be  counted  among 
the  complications.  Such  a  targeted  approach  to  the  review  of  in-hospital 
records  would  allow  accurate  identification  of  stroke  victims  without 
requiring  a  major  chart  review  effort.  The  state  of  Kentucky  made  the  only 
attempt  at  an  organized  monitoring  system  of  chart  review  for  the  purpose 
of  evaluating  endarterectomy  outcomes.^  No  screening  mechanism  such  as 
the  claims  data  was  established,  and  the  program  was  terminated  after  two 
years  due  to  its  financial  burden.”  By  screening  all  endarterectomy  cases  for 
bad  outcomes  with  the  claims  data,  only  a  small  fraction  of  their  charts 
would  need  to  be  reviewed. 

One  limitation  of  a  monitoring  program  would  be  that  individual 
hospitals  where  few  endarterectomies  are  performed  could  not  be  evaluated 
reliably.  While  some  low-volume  hospitals  may  have  better  results  than 
others,  the  small  number  of  cases  available  for  review  at  these  institutions 
makes  it  impossible  to  distinguish  with  certainty  the  few  with  excellent 
outcomes  from  those  with  worse  than  average  performance.  Where  research 
supports  a  volume-outcome  association,  regionalization  should  lead  to  better 
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outcomes,  as  long  as  the  hospitals  to  whom  patients  are  referred  have 
demonstrably  lower  complication  rates.  The  difficulty  in  evaluating 
outcomes  at  low-volume  hospitals  obviously  poses  a  problem  for  advocates 
of  continued  surgery  at  these  institutions.  From  the  patient's  perspective, 
however,  knowledge  of  the  risk  of  complications  at  a  specific  institution  is 
critical,  especially  for  an  elective  procedure  where  the  potential  benefit 
depends  upon  a  low  complication  rate.  The  Medicare  claims  files, 
supplemented  as  necessary  with  chart  review,  could  provide  an  efficient 
mechanism  for  improving  the  information  available  to  patients. 

Alternatively,  the  information  gleaned  from  chart  review  could  be  used  to 
direct  appropriateness  studies  in  the  fashion  proposed  by  Winslow  and 
colleagues.^ 

If  the  HCFA  adopts  this  method  for  surveillance  of  medical  care,  the 
agency  could  require  the  submission  of  appropriate  pieces  of  clinical 
information  for  various  medical  conditions  and  surgical  procedures.  Such 
regulations  would  undoubtedly  be  expensive  to  implement,  both  because  no 
standardized  format  currently  exists  and  because  health  care  providers 
already  feel  inundated  with  administrative  requests.  Furthermore,  new 
biases  would  be  introduced.  While  there  is  already  suggestion  that  the  DRG 
system  of  reimbursement  has  skewed  coding  to  grant  hospitals  higher 
remunerations  than  before,®^  the  systematized  use  of  claims  data  for 
evaluating  care  would  clearly  introduce  new  incentives  for  doctors  and 
hospitals  to  bias  the  reporting  of  their  data. 
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Several  areas  of  future  research  are  needed  to  complete  the  course  of  the 
work  reported  here.  First,  the  relationship  of  volume  to  complication  rates 
in  carotid  endarterectomy  needs  further  research.  Using  epidemiologic 
techniques,  this  study  shows  a  correlation  between  outcome  and  volume. 

The  logical  next  step  is  to  understand  the  cause  of  this  relationship.  Part  of 
this  finding  may  be  due  to  variation  in  case  mix.  If  the  finding  is  due  to 
true  differences  in  outcomes,  however,  the  variation  may  be  due  to  the 
surgeon's  technical  skill,  institutional  differences  such  as  post-operative  care, 
or  both.  One  recent  study  looked  at  these  two  components  of  surgical 
outcomes  for  several  different  procedures.’^  It  showed  that  for  some 
operations  the  institution  had  more  impact  and  for  others  the  surgeon  was 
the  key  influence  on  outcome.  Carotid  endarterectomy  was  not  considered 
by  that  research  team.  As  Medicare  makes  more  data  available  for  clinical 
review.  Part  B  claims  (physicians'  billings)  could  be  linked  to  the  already 
created  patient-specific  files,  and  the  impact  of  individual  surgeons'  volume 
could  be  evaluated. 

Similar  to  the  volume-outcome  relationship,  the  correlation  of  age  to 
complications  needs  to  be  examined.  It  is  likely  that  age  represents  a  proxy 
for  various  forms  of  physical  decline  or  disease  progression.  While  there  is 
a  relatively  high  risk  from  endarterectomy  in  the  elderly,  especially  the  very 
old,  a  large  majority  endure  the  surgery  without  complication.  A  more 
detailed  examination  of  what  conditions  cause  the  elderly  to  suffer  higher 
complication  rates  is  needed  in  order  to  stratify  this  group,  not  only  based 
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on  age,  but  also  by  using  pre-existing  medical  conditions.  The  claims  data 
cannot  perform  this  task,  as  it  identifies  comorbidities  poorly.  Only  when 
these  high-risk  conditions  are  identified  can  endarterectomy  be  performed 
safely  in  those  elderly  who  lack  these  poor  prognostic  signs. 

A  second  research  initiative  is  needed  to  further  evaluate  the  use  of 
claims  data,  not  only  as  it  relates  to  outcomes  of  carotid  endarterectomy  but 
also  as  it  applies  to  other  possible  uses  in  health  services  research.  A  well 
designed  study  of  the  quality  of  the  data  regarding  stroke  coding  and 
timing  is  needed.  Currently  only  the  pilot  project  examining  the  claims  data 
is  complete.  Other  researchers  such  as  Fisher  have  begun  work  in  this 
area.*^  As  claims  based  research  becomes  a  more  common  method  for 
evaluating  medical  and  surgical  outcomes,  a  greater  understanding  of  the 
data's  accuracy  and  of  the  appropriate  applications  of  the  technique  is 
necessary. 

A  third  area  of  future  investigation  relates  to  more  complete  reporting  of 
outcomes  in  the  setting  of  carotid  endarterectomy.  The  claims  data  can  be 
used  to  gather  information  on  surgical  morbidities  other  than  strokes.  This 
would  be  especially  useful  in  reviewing  post-operative  Mis,  since  this  is  a 
recognized  complication  of  the  operation  with  severe  consequences.  Using 
the  data  and  methods  reported  in  the  body  of  this  paper,  I  performed  an 
initial  examination  of  cardiac  complications  (including  MI,  unstable  angina, 
and  dysrhythmias).  I  found  a  3.2%  cardiac  complication  rate  at  30  days 
(data  not  reported  in  the  Results  section),  a  finding  consistent  with  the  work 
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of  others.^’'^  Additional  work  is  required  to  carefully  identify  each  of  these 
complications  and  to  examine  their  relationship  to  age  and  surgical  volume. 

At  present,  controversy  exists  over  the  role  of  carotid  endarterectomy  in 
the  prevention  of  stroke.  Given  the  uncertain  benefit  of  the  operation  in 
reducing  subsequent  strokes,  it  is  unreasonable  to  perform  it  in  patients  at 
high  risk  for  severe  complications.  This  study  demonstrates  significantly 
elevated  risk  in  two  settings—advanced  patient  age  and  low  hospital 
endarterectomy  volume.  These  correlations  raise  question  about  the 
appropriate  candidates  and  settings  for  the  operation,  and  future  research  is 
needed  to  clarify  the  causes  of  these  risks.  In  addition,  the  claims  data  can 
potentially  be  used  to  create  an  on-going  system  of  data  collection  and 
dissemination  to  inform  providers  and  patients  of  the  institution-specific  risk. 
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Table  2 


Death  Within  30  Days  of  Carotid  Endarterectomy 
Risk  and  Adjusted  Odds  Ratio  by  Patient  Age 


Patient 

Age 

Cases 

Deaths 

Risk 
(95%  C.I.) 

Adjusted  Odds  Ratio’ 
(95%  C.I.) 

65-69 

733 

8 

1.1% 

(0.5, 2.1) 

1.00 

70-74 

671 

19 

2.8% 

(1. 7,4.4) 

2.58 

(1.12,5.95) 

75-79 

473 

15 

3.2% 

(1.8,5.2) 

2.91 

(1.22,6.95) 

80  + 

212 

10 

4.7% 

(2.3,8.5) 

4.46 

(1.73,11.50) 

TOTAL 

2089 

52 

2.5% 

(1. 9,3.3) 

— 

Derived  from  multiple  logistic  regression  with  age,  sex,  surgical  volume, 
and  presence  or  absence  of  coronary  artery  disease  as  covariables.  The  65  to 
69  year  old  group  is  the  reference. 
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Table  3 


Death  Within  30  Days  of  Carotid  Endarterectomy 
Risk  and  Adjusted  Odds  Ratio  by  Procedure  Volume 


Adjusted 


Hospital 

Volume’ 

Cases 

Deaths 

Risk 

(95%  C.I.) 

Odds  Ratio" 
(95%  C.I.) 

Over  40 

457 

5 

1.1% 

(0.3,2.5) 

1.00 

21-40 

561 

15 

2.7% 

(1.5,4.4) 

2.63 

(0.94,7.32) 

6-20 

913 

27 

3.0% 

(2.0,4.3) 

2.90 

(1.10,7.62) 

5  or 
less 

158 

5 

3.2% 

(1. 0,7.2) 

3.14 

(0.89,11.06) 

TOTAL 

2089 

52 

2.5% 

(19,3.3) 

— - 

*  Volume  defined  as  the  number  of  carotid  endarterectomies 
performed  per  year  on  the  Medicare  population  at  each 
hospital. 

"Derived  from  multiple  logistic  regression  with  age,  sex, 
surgical  volume,  and  presence  or  absence  of  coronary 
artery  disease  as  covariables.  The  over  40  operations 
category  is  the  reference. 
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Table  4 


Death  or  Stroke  within  30  Days  of  Carotid  Endarterectomy 
Risk  and  Adjusted  Odds  Ratios  (AOR)  by  Patient  Age 


Death  or  Definite  Stroke  Death  or  Possible  Stroke 


Patient 

Age 

Risk 

(95%  C.I.) 

AOR” 
(95%  C.I.) 

Risk 

(95%  C.I.) 

AOR” 
(95%  C.I.) 

65-69 

2.7% 

(1.7,4.2) 

1.00 

8.7% 

(6.8,11.0) 

1.00 

70-74 

4.3% 

(2.9,6.2) 

1.59 

(0.89,2.84) 

13.1% 

(10.7,15.9) 

1.61 

(1.15,2.27) 

75-79 

5.1% 

(3.3,7.5) 

1.87 

(1.02,3.44) 

12.7% 

(9.8,16.0) 

1.54 

(1.06,2.25) 

80  + 

6.6% 

(3.7,10.8) 

2.54 

(1.25,5.14) 

17.0% 

(12.2,22.7) 

2.21 

(1.42,3.45) 

TOTAL 

4.2% 

(3.3,5.1) 

— 

11.9% 

(10.5,13.3) 

— 

See  Table  2  for  numbers  of  patients  considered  in  each  category. 

Derived  from  multiple  logistic  regression  with  age,  sex,  surgical  volume, 
and  presence  or  absence  of  coronary  artery  disease  as  covariables.  The  65 
to  69  year  old  group  is  the  reference. 
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Table  5 


Death  or  Stroke  within  30  Days  of  Carotid  Endarterectomy 
Risk  and  Adjusted  Odds  Ratios  (AOR)  by  Procedure  Volume 


Death  or  Definite  Stroke  Death  or  Possible  Stroke 


Hospital 

Volume” 

Risk 

(95%  C.I.) 

AOR’ 
(95%  C.I.) 

Risk 

(95%  C.I.) 

AOR' 
(95%  C.I.) 

Over  40 

2.0% 

(0.9,3.7) 

1.00 

9.2% 

(6.7,12.2) 

1.00 

21-40 

4.6% 

(3.0,6.7) 

2.52 

(1.16,5.44) 

10.9% 

(8.4,13.7) 

1.19 

(0.78,1.80) 

6-20 

4.7% 

(3.4,6.3) 

2.55 

(1.23,5.28) 

13.0% 

(10.9,15.4) 

1.46 

(1.00,2.12) 

5  or 
less 

5.7% 

(2.6,10.5) 

3.13 

(1.22,8.07) 

16.5% 

(11.0,23.2) 

1.96 

(1.15,3.32) 

TOTAL 

4.2% 

(3.3,5.1) 

— 

11.9% 

(10.5,13.3) 

— 

Derived  from  multiple  logistic  regression  with  age,  sex,  surgical  volume, 
and  presence  or  absence  of  coronary  artery  disease  as  covariables.  The 
first  level  of  each  category  is  the  reference. 

Volume  defined  as  the  number  of  carotid  endarterectomies  performed  per 
year  on  the  Medicare  population  at  each  hospital.  See  Table  3  for  the 
number  of  patients  considered  in  each  category. 
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Table  6 


Number  of  Hospitals  Identified  as  Exceeding 
Outcome  Thresholds  by  Surgical  Volume 


Hospital 

Volume’ 

Number  of 
Hospitals  in 
Group 

Hospitals  Identified  with 
Definite  Stroke  Plus  Death 
Rates  Significantly 
Above  Expected  (p  <  .10)” 

1  to  5 

50 

6 

6  to  20 

65 

10 

21  to  40 

17 

6 

Over  40 

7 

0 

TOTAL 

139 

22 

Volume  defined  as  the  number  of  carotid  endarterectomies  performed  per 
year  on  the  Medicare  population  at  each  hospital. 

For  each  hospital,  the  expected  proportion  of  definite  strokes  plus  deaths 
is  assumed  to  be  .0197,  the  proportion  that  occurred  in  the  high-volume 
hospitals.  Given  this  proportion,  the  probability  that  a  hospital  would 
have  at  least  as  many  strokes  plus  deaths  as  actually  reported  is 
computed.  A  hospital  where  ^is  probability  is  .10  or  less  (using  a  chi- 
square  test)  is  considered  to  have  a  significantly  high  rate.  The  number 
of  definite  strokes  plus  deaths  is  assumed  to  have  a  binomial  distribution. 
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Table  7 


Complication  Rates  of  High  Surgical  Volume  Hospitals 


Hospital 

A 

B 

C 

D 

E 

F 

G 


Death  (%) 


Death  or 
Definite 
Stroke  (%) 


Death  or 
Possible 
Stroke  (%) 


0.0 

0.0 

9.4 

0.0 

0.0 

13.0 

0.0 

3.1 

6.2 

1.0 

1.0 

6.1 

1.5 

4.6 

16.9 

1.9 

1.9 

5.8 

3.4 

3.4 

8.5 

0.0  -  3.4 

0.0  -  4.6 

5.8  -  16.9 

RANGE 
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APPENDIX  A 

ICD-9-CM  CODES  USED  TO  IDENTIFY  STROKES 


I.  Codes  included  in  stroke  identification,  with  headings  for  types  of  stroke 
code. 


ICD-9-CM  CODE 

1.  Symptom  of  stroke 

342.0 

342.1 

342.9 

344.3 

344.4 

344.5 
784.3 

2.  Pathologic  definition  of  stroke 

434 

434.0 

434.1 

434.9 

3.  Acute  cerebrovascular  disease 

436 


DEFINITION 


Flaccid  hemiplegia 
Spastic  hemiplegia 
Hemiplegia,  unspecified 
Monoplegia  of  lower  limb 
Monoplegia  of  upper  limb 
Unspecified  monoplegia 
Aphasia 


Occlusion  of  cerebral  arteries 
Cerebral  thrombosis 
Cerebral  embolism 

Cerebral  artery  occlusion,  unspecified 


Acute,  but  ill-defined  cerebrovascular 
disease 


4.  Late  effects  of  cerebrovascular  disease 

438  Late  effects  of  cerebrovascular  disease 

5.  Surgical  complication 

997.0  Central  nervous  system  complication  of 

surgical  or  medical  care 
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IL  Codes  considered  and  rejected  for  use  in  identification  of  strokes. 

Quadriplegia,  paraplegia,  diplegia,  paralysis  (unspecified),  anoxic  brain 
damage,  encephalopathy  (unspecified),  all  cranial  nerve  disorders,  retinal 
vascular  occlusion,  intracerebral  hemorrhage,  other  unspecified  intracranial 
hemorrhage,  occlusion  and  stenosis  of  precerebral  arteries,  transient  cerebral 
ischemia,  other  generalized  ischemic  cerebrovascular  disease,  coma,  stupor, 
syncope,  collapse,  dizziness,  abnormality  of  gait,  lack  of  coordination, 
transient  limb  paralysis,  disturbance  of  skin  sensation,  voice  disturbance, 
other  speech  disturbance  (including  dysarthria,  dysphasia,  and  slurred 
speech),  other  symbolic  dysfunction  (including  alexia,  agraphia,  acalculia, 
agnosia,  and  apraxia). 
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APPENDIX  B 

DEFINITION  OF  POST-OPERATIVE  STROKES 
I.  Definite  Post-operative  Strokes 

TYPE  OF  STROKE  CODE  CRITERIA 


1.  Symptom  of  stroke 

Code  must  appear  on  post-index 
admission  and  may  not  appear  on 
index  or  prior  admission 

2.  Pathologic  definition 

Code  must  appear  on  post-index 
admission  and  may  not  appear  on 
index  or  prior  admission 

3.  Acute  CV  disease 

Code  must  appear  on  post-index 
admission 

4.  Late  effects  of  CV  disease 

Code  must  appear  on  post-index  and 
may  not  appear  on  index  or  prior 
admission 

5.  Surgical  complication 

Code  must  appear  on  index  or  post¬ 
index  admission 

II.  Possible  Post-operative  Strokes 


TYPE  OF  STROKE  CODE 

CRITERIA 

1.  Symptom  of  stroke 

Code  must  not  appear  prior  to  index 
admission 

2.  Pathologic  definition 

Code  must  not  appear  prior  to  index 
admission 

3.  Acute  CV  disease 

Code  may  appear  prior  to  index 
admission  as  long  as  it  also  appears  on 
index  or  post-index  admission 
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Notes: 


1.  The  definitions  of  the  "Types  of  Stroke  Codes"  are  found  in  Appendix  A. 

2.  In  order  to  qualify  as  a  stroke  victim,  a  patient  had  to  meet  any  one  of 
the  entries'  criteria;  only  patients  who  met  these  criteria  were  defined  as 
having  had  a  post-operative  stroke. 

3.  Patients  who  met  the  criteria  for  both  possible  and  definite  post-operative 
stroke  were  counted  only  among  the  definite  post-operative  stroke  victims. 
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APPENDIX  C 

ICD-9-CM  CODES  USED  TO  DEFINE  CLINICAL  COVARIABLES* 


Condition 

ICD-CM-9  codes  present 
on  admissions  occurring 
in  previous  6  months 

ICD-CM-9  codes 
on  index 
admission” 

Coronary  artery 
disease 

410,  411.1,  411.8, 

412,  413,  414,  429.2 

412,  413,  414, 
429.2 

Valvular 

disease 

394  to  397,  421  to 

422.9,  424 

394  to  397,  421 
to  422.9,  424 

Congestive 
heart  failure 

402.01,  402.11, 

402.91,  428,  429.3 

none 

Pulmonary 

disease 

466,  480  to  486, 

490  to  496 

490  to  496 

Renal  disease 

580  to  586 

580  to  586, 
exclude  584 

Diabetes 

250  to  250.91 

250  to  250.91, 
except  250.10, 
250.20,  250.30 

'  Clinical  condition  was  coded  as  present  if  the  specified  codes  were 
recorded  on  index  or  preceding  admission. 

’*Diagnostic  codes  that  represent  conditions  that  could  have  occurred  as  a 
complication  of  the  endarterectomy  are  not  used  to  create  the  clinical 
covariable. 
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APPENDIX  D 


A  STUDY  OF  THE  SENSITIVITY  AND  SPECIFICITY  OF  THE 
CODING  OF  STROKE  IN  THE  MEDICARE  CLAIMS  DATA 


Introduction 


A  small,  unfunded  study  of  the  validity  of  the  clinical  information 
catalogued  in  the  Medicare  claims  data  was  performed. 

Methods 


The  medical  records  were  requested  for  101  patients  who  received 
carotid  endarterectomy  in  one  of  14  hospitals  in  New  Hampshire  or 
Vermont  between  April  1,  1984  and  June  30,  1985.  The  sample  chosen 
included  all  patients  operated  on  in  the  two  states  who  had  a  complication 
according  to  the  Medicare  claims  data  (n  =  27).  Of  those  patients  whose 
outcomes  were  benign  according  to  the  claims  data,  all  patients  who  were 
rehospitalized  within  thirty  days  were  sampled  (n  =  21).  The  rest  of  the 
sample  was  drawn  randomly  from  those  patients  without  complication 
according  to  the  claims  data  (see  Table  1). 
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TABLE  1:  CHART  ACQUISITION 


complications 

readmit 

by  claims  data 
(liberal  method) 

w/in  30  d 

neither 

total 

charts 

requested 

27 

21 

53 

101 

charts 

received 

17 

10 

38 

65 

The  medical  records  department  at  each  hospital  was  asked  to 
photocopy  the  discharge  abstract,  discharge  summary,  physicians'  and 
nurses'  notes,  operative  report,  laboratory  reports,  and  reports  of 
radiographic  studies  for  identified  patients  during  specified  time  intervals. 
Information  was  sought  from  the  hospitalization  for  the  first  endartectomy 
("index  admission")  plus  all  subsequent  admissions  to  any  New  England 
hospital  within  30  days  of  the  index  procedure. 

Seven  institutions  responded  to  the  initial  inquiry  or  a  follow-up 
request.  The  rest  did  not  respond  to  repeated  queries  or  refused  to  comply 
due  to  issues  of  patient  confidentiality  (although  anonymity  for  both 
hospitals  and  patients  was  pledged).  The  primary  responders  were  the 
four  hospitals  in  New  Hampshire  with  the  largest  Medicare  caseloads  in  the 
state.  Sixty  five  charts  were  collected.  Six  cases  were  disqualified  from 
study  because  the  medical  record  of  a  follow-up  admission  within  30  days 
of  index  operation  was  unavailable.  One  patient  was  excluded  from  the 
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study  because  the  photocopied  version  provided  for  our  review  lacked  the 
section  of  the  chart  containing  physicians'  notes. 

A  stroke  was  defined  as  any  hemispheric  event  causing  unilateral 
weakness,  paralysis,  or  changes  in  sensation,  aphasia  of  any  type,  or 
monocular  blindness.  Symptoms  of  vertebral-basilar  strokes,  including 
vertigo,  ataxia  and  syncope,  were  not  included  in  our  definition  of  stroke 
because  we  we  interested  only  in  the  manifestations  of  carotid  artery 
disease.  Symptoms  that  resolved  in  less  than  one  week  were  considered  to 
be  transient  ischemic  attacks  (TIAs)  or  reversible  ischemic  neurologic  deficits 
(REvJDs),  and  were  not  counted  as  strokes.  Complications  to  vocal  cords 
were  considered  to  be  the  result  of  local  damage  to  the  recurrent  laryngeal 
nerve,  and  were  not  counted  as  strokes.  If  a  patient  had  neurologic 
impairment  prior  to  surgery  that  was  observed  to  be  worse  post-operatively, 
it  was  considered  a  new  stroke.  In  order  for  any  possible  stroke  symptom 
to  identify  a  patient  as  having  had  a  stroke,  it  had  to  be  reported  on  two 
separate  occasions  in  the  physicians'  and/or  nurses'  notes,  and  at  least  one 
time  the  note  had  to  come  from  an  MD.  Deaths  were  recorded  regardless 
of  their  cause. 

The  charts  were  reviewed  without  knowledge  of  the  outcomes 
reported  for  each  patient  in  the  claims  data. 


53 


Results 


Of  the  58  patients  whose  charts  were  reviewed,  one  never  had  an 
endartectomy.  That  patient  received  an  intracranial-extracranial  bypass 
operation.  His  results  were  not  included  in  subsequent  analyses.  Seven  of 
the  patients  suffered  a  post-operative  stroke  or  died. 

The  information  gained  from  chart  review  was  used  as  a  standard  for 
comparison  with  the  outcomes  generated  by  the  claims  data.  The  sensitivity 
and  specificity  and  positive  predictive  value  of  the  Medicare  claims  data  to 
report  post-operative  complications  was  computed  using  two  different 
measures  of  poor  outcomes:  death  plus  definite  strokes  (conservative 
algorithm),  and  death  plus  all  possible  strokes  (liberal  algorithm).  The  two- 
by-two  tables  of  these  algorithms  are  provided  below. 


TABLE  2 

2X2  TABLE  EVALUATING  CLAIMS  DATA 
FOR  THE  REPORTING  OF  STROKE  OR  DEATH 

CONSERVATIVE  ALGORHYTHM  FOR  STROKE 

CHART  REVIEW 


CLAIMS  +4  2 

DATA 

4  47 


Sensitivity  =  .50  (Cl:  .15,  .85) 

Specificity  =  .96  (Cl:  .90,  .99) 

Positive  predictive  value  =  .67  (Cl:  .29,  .99) 
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TABLE  3 

2X2  TABLE  EVALUATING  CLAIMS  DATA 
FOR  THE  REPORTING  OF  STROKE  OR  DEATH 

LIBERAL  ALGORHYTHM  FOR  STROKE 


CHART  REVIEW 


CLAIMS  +  7  10 

DATA 

1  39 


Sensitivity  =  .88  (Cl:  .65,  .99) 

Specificity  =  .80  (Cl:  .68,  .91) 

Positive  predictive  value  =  .41  (Cl:  .18,  .64) 


Comment 


This  small  study  demonstrates  that  the  claims  data  can  be  used  to 
report  outcomes  of  stroke  with  a  specificity  of  at  least  90%.  A  second 
finding  of  interest  is  that  by  using  the  more  liberal  definition  of  stroke,  it 
appears  that  the  claims  data  can  be  used  to  screen  endarterectomy  surgery 
for  bad  outcomes.  The  sensitivity  of  the  claims  data,  when  the  broader 
definition  of  stroke  is  used,  was  88%.  Follow-up  chart  review  is  necessary 
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to  determine  which  of  these  patients  had  surgical  complications. 

Nonetheless,  only  a  small  subset  of  the  total  population  receiving 
endarterectomy  would  need  in-depth  review  of  their  medical  records  in 
order  to  monitor  complications  of  the  surgery. 

This  was  a  pilot  study  to  check  the  validity  of  the  reporting  of  stroke 
and  death  in  the  Medicare  claims  data.  Because  there  were  few  endpoints 
in  this  sample,  the  confidence  interval,  especially  surrounding  sensitivity,  are 
large.  As  a  result,  conclusions  cannot  be  reached  with  certainty,  and  a 
larger  study  is  needed. 
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